Using backpropagation to train a dialog system

ABSTRACT

Techniques described herein use backpropagation to train one or more machine learning (ML) models of a dialog system. For instance, a method includes accessing seed data that includes training tuples, where each training tuple comprising a respective logical form. The method includes converting the logical form of a training tuple to a converted logical form, by applying to the logical form a text-to-speech (TTS) subsystem, an automatic speech recognition (ASR) subsystem, and a semantic parser of a dialog system. The method includes determining a training signal by using an objective function to compare the converted logical form to the logical form. The method further includes training the TTS subsystem, the ASR subsystem, and the semantic parser via backpropagation based on the training signal. As a result of the training by backpropagation, the machine learning models are tuned work effectively together within a pipeline of the dialog system.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority to U.S. Provisional ApplicationSer. No. 62/898,680 for “Techniques for Using Backpropagation to Train aDialog System,” filed Sep. 11, 2019, which is incorporated by referenceherein in its entirety.

TECHNICAL FIELD

The present disclosure relates to dialog systems and, more particularly,to techniques for using backpropagation to train machine learning modelsof a dialog system, for instance, where that training is based on actualpredictions made by machine learning models in a workflow pipeline ofthe dialog system, such that the machine learning models learn toimplicitly correct errors made within the dialog system.

BACKGROUND

An increasing number of devices now enable users to interact with thedevices directly using voice or spoken speech. For example, a user canspeak to such a device in a natural language, and in doing so, the usercan ask a question or make a statement requesting an action to beperformed. In response, the device performs the requested action orresponds to the user's question using audio output. Since interactingdirectly using voice is a more natural and intuitive way for humans tocommunicate with their surroundings, the popularity of such speech-basedsystems is growing at an astronomical rate.

BRIEF SUMMARY

The present disclosure relates to techniques for using backpropagation(i.e., backward propagation of errors) to train one or more machinelearning models of a dialog system. Specifically, such machine learningmodels, also referred to herein as models, may include one or more of atext-to-speech (TTS) subsystem, an automatic speech recognition (ASR)subsystem, and a semantic parser subsystem. As a result of suchtraining, the models may be tuned work effectively within a pipeline ofthe dialog system.

A training system according to some embodiments utilizes seed data as abasis for training various models in the dialog system. In someembodiments, the seed data includes a set of tuples, each tupleincluding an original utterance and a corresponding original logicalform that represents the original utterance. In some embodiments, thetraining system includes a conversion subsystem, which incorporates oneor more models selected from the dialog system. The conversion subsystemperforms a sequence of one, two, or more conversions. For each tuple inthe seed data, the conversion subsystem of the training system convertsthe tuple to a converted tuple, and the training system may then comparethe each converted tuple to the corresponding tuple from the seed datato determine how to update the machine learning models participating inthe conversion subsystem so as to improve the accuracy of conversions.

Specifically, some embodiments of the conversion subsystem utilize, andthus include, a TTS subsystem, an ASR subsystem, and a semantic parsersubsystem selected from a dialog system. The conversion subsystem mayalso utilize, and thus include, an inverse sequence-to-sequence(seq2seq) model that is the inverse of the semantic parser subsystem.For each tuple in the seed data, the conversion subsystem applies theinverse sequence-to-sequence (seq2seq) model to the original logicalform of the tuple to cause the inverse seq2seq model to determine asecond utterance. The conversion subsystem applies the TTS subsystem tothe second utterance to cause the TTS subsystem to determine audio data.The conversion subsystem applies the ASR subsystem to the audio data tocause the ASR subsystem to determine a third utterance. The conversionsubsystem applies the semantic parser subsystem to the third utteranceto cause the semantic parser subsystem to determine a converted logicalform.

In some embodiments, for each tuple of the training data, the trainingsystem applies an objective function to determine a degree of differencebetween the converted logical form and the original logical form fromthe tuple, which are ideally the same. The training system may use theresult of the objective function to train the inverse seq2seq model, theTTS subsystem, the ASR subsystem, and the semantic parser subsystem byway of backpropagation. As a result, the TTS subsystem, the ASRsubsystem, and the semantic parser subsystem may be tuned to work moreeffectively together within the dialog system.

The foregoing, together with other features and embodiments will becomemore apparent upon referring to the following specification, claims, andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example of a dialog system 100 utilizing anautomatic speech recognition subsystem, a semantic parser, and atext-to-speech subsystem trained by way of backpropagation, according tocertain embodiments described herein.

FIG. 2 is a diagram of an example of a training system for training oneor more machine learning models of the dialog system, according tocertain embodiments described herein.

FIG. 3 is a diagram of an example of a method of using backpropagationto train one or more machine learning models of the dialog system,according to certain embodiments described herein.

FIG. 4 is a diagram of another example of the training system fortraining one or more machine learning models of the dialog system,according to certain embodiments described herein.

FIG. 5 is a diagram of another example of a method of usingbackpropagation to train one or more machine learning models of thedialog system, according to certain embodiments described herein.

FIG. 6 is a diagram of a distributed system for implementing certainembodiments described herein.

FIG. 7 is a diagram of a cloud-based system environment in whichtraining machine learning models of the dialog system may be offered atleast in part as a cloud service, according to certain embodimentsdescribed herein.

FIG. 8 is a diagram of an example of a computer system that can be usedto implement certain embodiments described herein.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofcertain embodiments. However, it will be apparent that variousembodiments may be practiced without these specific details. The figuresand description are not intended to be restrictive. The word “exemplary”is used herein to mean “serving as an example, instance, orillustration.” Any embodiment or design described herein as “exemplary”or as an “example” is not necessarily to be construed as preferred oradvantageous over other embodiments or designs.

A voice-enabled system that is capable of having a dialog with a uservia speech inputs and audio outputs, also referred to as voice outputs,can come in various forms. For example, such a system may be provided asa stand-alone device, as a digital or virtual assistant, as avoice-capable service, or the like. In each of these forms, the systemis capable of receiving speech inputs, understanding the speech inputs,generating responses or taking actions responsive to the speech inputs,and outputting the responses using audio outputs. In certainembodiments, the dialog functionality in such a voice-enabled system isprovided by a dialog system or infrastructure (“dialog system”). Thedialog system is configured to receive speech inputs, interpret thespeech inputs, maintain a dialog, possibly perform or cause one or moreactions to be performed based on interpretations of the speech inputs,prepare appropriate responses, and output the responses to the userusing audio output.

Conventionally, a dialog system includes various machine learning (ML)models, such as an automatic speech recognition (ASR) subsystem, asemantic parser subsystem, and a text-to-speech (TTS) subsystem. TheseML models are typically trained with clean data, i.e., data that is notthe output of a different component of the dialog system. As a result,the ML models learn to handle clean data, rather than data that hasalready been processed and likely has had errors introduced. Forinstance, if the ASR subsystem makes an error in translating speechinput to text, then that error is passed along to the semantic parser inthe form of an inaccurate utterance. The semantic parser subsystem thenproduces a logical form based on the inaccurate utterance. Analogously,if the semantic parser subsystem makes an error, that error is passedalong to the dialog manager subsystem, which generates response text asa reply to the original speech input based on a propagation of errorsthroughout the pipeline of the dialog system. The TTS subsystem thengenerates speech output based on the response text and, thus, indirectlybased on one or more errors in the dialog system. Such errors candramatically diminish the user experience when a user seeks a dialogwith the dialog system.

Embodiments described herein provide improved techniques for trainingone or more ML models of the dialog system. In some embodiments, atraining system described herein utilizes backpropagation (i.e.,backward propagation of errors) to train such ML models. For instance, atraining system described herein utilizes a set of seed data includingvarious training tuples, in which each training tuple includes arespective utterance and a corresponding logical form. The trainingsystem uses one or more ML models of a dialog system to convert atraining tuple of the seed data to one or more other formats, such thatthe ML models together determine a converted training tuple. Theconverted training tuple is thus a representation of the training tupleafter the application of ML models of the dialog system. Ideally,because the ML models translate data from one format to another (e.g.,from an utterance to a logical form representing the utterance), theconverted training tuple should match the training tuple. For instance,if the ML models convert the logical form of the training tuple to anutterance, to audio data, to a second utterance, and then to a secondlogical form, the second logical form should ideally be the same as thelogical form from the seed data. In some embodiments, the trainingsystem compares the converted training tuple to the training tuple, andthe training system uses an error between the converted training tupleand the training tuple as a training signal with which to train the MLmodels used in the conversion via backpropagation.

Thus, through backpropagation, each ML model trained as described hereinmay be tuned to work with other ML models of the dialog system. Theresult is a dialog system with ML models that are tuned based on errorsexpected within the pipeline of the dialog system, so as to reduce sucherrors over the entire pipeline of the dialog system during operation.

FIG. 1 is a diagram of an example of a dialog system 100 utilizing anASR subsystem 108, a semantic parser 114, and a TTS subsystem 120trained by way of backpropagation, according to certain embodimentsdescribed herein. The dialog system 100 is configured to receive speechinputs 104, also referred to as voice inputs, from a user 102. Thedialog system 100 may then interpret the speech inputs 104. The dialogsystem 100 may maintain a dialog with a user 102 and may possiblyperform or cause one or more actions to be performed based uponinterpretations of the speech inputs 104. The dialog system 100 mayprepare appropriate responses and may output the responses to the userusing voice or speech output, also referred to as audio output. Thedialog system 100 is a specialized computing system that may be used forprocessing large amounts of data potentially using a large number ofcomputer processing cycles. The numbers of devices depicted in FIG. 1are provided for illustrative purposes. Different numbers of devices maybe used. For example, while each device, server, and system in FIG. 1 isshown as a single device, multiple devices may be used instead.

In certain embodiments, the processing performed by the dialog system100 is implemented by a pipeline of components or subsystems, includinga speech input component 105; a wake-word detection (WD) subsystem 106;an ASR subsystem 108, also referred to as an ASR 108; a natural languageunderstanding (NLU) subsystem 110, which includes a named entityrecognizer (NER) subsystem 112 and a semantic parser subsystem 114; adialog manager (DM) subsystem 116; a natural language generator (NLG)subsystem 118; a TTS subsystem 120; and a speech output component 124.The subsystems listed above may be implemented only in software (e.g.,using code, a program, or instructions executable by one or moreprocessors or cores), in hardware, or in a combination of hardware andsoftware. In certain implementations, one or more of the subsystems maybe combined into a single subsystem. Additionally or alternatively, insome implementations, the functions described herein as performed by aparticular subsystem may be implemented by multiple subsystems.

The speech input component 105 includes hardware and software configuredto receive speech input 104. In some instances, the speech inputcomponent 105 may be part of the dialog system 100. In some otherinstances, the speech input component 105 may be separate from and becommunicatively coupled to the dialog system 100. The speech inputcomponent 105 may, for example, include a microphone coupled to softwareconfigured to digitize and transmit speech input 104 to the wake-worddetection subsystem 106.

The wake-word detection (WD) subsystem 106 is configured to listen forand monitor a stream of audio input for input corresponding to a specialsound or word or set of words, referred to as a wake-word. Upondetecting the wake-word for the dialog system 100, the WD subsystem 106is configured to activate the ASR subsystem 108. In certainimplementations, a user may be provided the ability to activate ordeactivate the WD subsystem 106 (e.g., by pushing a button) to cause theWD subsystem 106 to listen for or stop listening for the wake-word. Whenactivated, or when operating in active mode, the WD subsystem 106 isconfigured to continuously receive an audio input stream and process theaudio input stream to identify audio input, such as speech input 104,corresponding to the wake-word. When audio input corresponding to thewake-word is detected, the WD subsystem 106 activates the ASR subsystem108.

As described above, the WD subsystem 106 activates the ASR subsystem108. In some implementations of the dialog system 100, mechanisms otherthan wake-word detection may be used to trigger or activate the ASRsubsystem 108. For example, in some implementations, a push button on adevice may be used to trigger the ASR subsystem 108 without needing awake-word. In such implementations, the WD subsystem 106 need not beprovided. When the push button is pressed or activated, the speech input104 received after the button activation is provided to the ASRsubsystem 108 for processing. Additionally or alternatively, in someimplementations, the ASR subsystem 108 may be activated upon receivingan input to be processed.

The ASR subsystem 108 is configured to receive and monitor speech input104 after a trigger or wake-up signal (e.g., a wake-up signal may besent by the WD subsystem 106 upon the detection of the wake-word in thespeech input 104, or the wake-up signal may be received upon theactivation of a button) and to convert the speech input 104 to text. Aspart of its processing, the ASR subsystem 108 performs speech-to-textconversion. The speech input 104 may be in a natural language form, andthe ASR subsystem 108 is configured to generate the correspondingnatural language text in the language of the speech input 104. Thiscorresponding natural language text is referred to herein as anutterance. For instance, the speech input 104 received by the ASRsubsystem 108 may include one or more words, phrases, clauses,sentences, questions, or the like. The ASR subsystem 108 is configuredto generate an utterance for each spoken clause and feed the utterancesto the NLU subsystem 110 for further processing.

The NLU subsystem 110 receives utterances generated by the ASR subsystem108. The utterances received by the NLU subsystem 110 from the ASRsubsystem 108 may include text utterances corresponding to spoken words,phrases, clauses, or the like. The NLU subsystem 110 translates eachutterance, or a series of utterances, to a corresponding logical form.

In certain implementations, the NLU subsystem 110 includes a namedentity recognizer (NER) subsystem 112 and a semantic parser subsystem114. The NER subsystem 112 receives an utterance as input, identifiesnamed entities in the utterance, and tags the utterance with informationrelated to the identified named entities. The tagged utterances are thenfed to the semantic parser subsystem 114, which is configured togenerate a logical form for each tagged utterance, or for a series oftagged utterances. The logical form generated for an utterance mayidentify one or more intents corresponding to the utterance. An intentfor an utterance identifies an objective of the utterance. Examples ofintents include “order pizza” and “find directions.” An intent may, forexample, identify an action that is requested to be performed. Inaddition to intents, a logical form generated for an utterance may alsoidentify slots, also referred to as parameters or arguments, for anidentified intent. For example, for the speech input “I'd like to ordera large pepperoni pizza with mushrooms and olives,” the NLU subsystem110 can identify the intent order pizza. The NLU subsystem can alsoidentify and fill slots, e.g., pizza_size (filled with large) andpizza_toppings (filled with mushrooms and olives). The NLU subsystem 110may use machine learning based techniques, rules, which may be domainspecific, or a combination of machine learning techniques and rules togenerate the logical forms. The logical forms generated by the NLUsubsystem 110 are then fed to the DM subsystem 116 for furtherprocessing.

As shown in FIG. 1, in some embodiments, a training system 150 describedherein trains one or more ML models of the dialog system 100, such asthe ASR subsystem 108, the semantic parser subsystem 114, and the TTSsubsystem 200. In some embodiments, as described in detail below, thetraining system 150 incorporates the one or more ML models into aconversion subsystem, which converts seed data into converted seed data.The training system 150 determines an error between the converted seeddata and the seed data, and the training system 150 utilizes that errorto train the one or more ML models via backpropagation. As a result, theone or more ML models are tuned to work together to reduce thepropagation of errors through the dialog system 100.

The DM subsystem 116 is configured to manage a dialog with the userbased on logical forms received from the NLU subsystem 110. As part ofthe dialog management, the DM subsystem 116 is configured to trackdialog states, initiate the execution of or itself execute one of moreactions or tasks, and determine how to interact with the user. Theseactions may include, for example, querying one or more databases,producing execution results, or other actions. For example, the DMsubsystem 116 is configured to interpret the intents identified in thelogical forms received from the NLU subsystem 110. Based on theinterpretations, the DM subsystem 116 may initiate one or more actionsthat it interprets as being requested by the speech inputs 104 providedby the user. In certain embodiments, the DM subsystem 116 performsdialog-state tracking based on current and past speech inputs 104 andbased on a set of rules (e.g., dialog policies) configured for the DMsubsystem 116. These rules may specify the different dialog states,conditions for transitions between states, actions to be performed whenin a particular state, or the like. These rules may be domain specific.The DM subsystem 116 also generates responses to be communicated back tothe user involved in the dialog. These responses may be based uponactions initiated by the DM subsystem 116 and their results. Theresponses generated by the DM subsystem 116 are fed to the NLG subsystem118 for further processing.

The NLG subsystem 118 is configured to generate natural language textscorresponding to the responses generated by the DM subsystem 116. Thetexts may be generated in a form that enables them to be converted tospeech by the TTS subsystem 120. The TTS subsystem 120 receives thetexts from the NLG subsystem 118 and converts each of them to speech orvoice audio, which may then be output as audio to the user via an audioor speech output component 124 of the dialog system (e.g., a speaker, orcommunication channel coupled to an external speaker). In someinstances, the speech output component 124 may be part of the dialogsystem 100. In some other instances, the speech output component 124 maybe separate from and communicatively coupled to the dialog system 100.

As described above, the various subsystems of the dialog system 100working in cooperation provide the functionality that enables the dialogsystem 100 to receive speech inputs 104 and to respond using speechoutputs 122 and, thereby, to maintain a dialog with a user using naturallanguage speech. The various subsystems described above may beimplemented using a single computer system or using multiple computersystems working cooperatively. For example, for a device implementingthe voice-enabled system, the subsystems of the dialog system 100described above may be implemented entirely on the device with which theuser interacts. In some other implementations, some components orsubsystems of the dialog system 100 may be implemented on the devicewith which the user interacts, while other components may be implementedremotely from the device, possibly on some other computing devices,platforms, or servers.

FIG. 2 is a diagram of an example of the training system 150, which isconfigured to train one or more ML models of a dialog system 100,according to certain embodiments described herein. In some embodiments,the training system 150 is implemented as a computing device or portionthereof, such as a server. The training system 150 may be implemented ashardware, software, or a combination of both. For instance, the trainingsystem 150 may be a specialized hardware device or program code, or acombination of both. For instance, the operations described herein asbeing performed by the training system 150 may be embodied in programcode implementing the training system 150, where such program code isexecutable by one or more processing units.

As described above, the dialog system 100 includes various ML models inits pipeline, or workflow. More specifically, these models may includean ASR 108, a semantic parser subsystem 114, and a TTS subsystem 120,potentially in addition to others. As shown in FIG. 2, the trainingsystem 150 may be configured to train one or more of such ML models thatare selected from the dialog system 100 and incorporated into aconversion subsystem 240 of the training system 150.

As described above, various ML models of the dialog system 100 areconfigured to translate, or convert, data from one format to another.For instance, a user provides speech input 104 to the dialog system 100,such as by speaking. In some embodiments of the dialog system 100, theASR 108 translates the speech input 104 into an utterance 225, which thesemantic parser subsystem 114 translates into a logical form 235, whichthe dialog manager subsystem 116 processes to determine a response,which the TTS subsystem 120 translates into speech output 122 responsiveto the speech input 104. By the conversion subsystem 240 of the trainingsystem 150, seed data 210 is translated one or more times using one ormore of these ML models. The training system 150 may compare the resultof such translations, as performed by the conversion subsystem 240, tothe original seed data 210 to train the ML models that participate inthe conversion subsystem 240.

As shown in FIG. 2, the training system 150 may have access to a set ofseed data 210, also referred to as training data, which may include aset of tuples. Each tuple in the seed data 210 may include an utterance225 and a corresponding logical form 235 (i.e., an expression of theutterance in the language of logical forms). Generally, the logical form235 in a given tuple may be a structured translation of thecorresponding utterance 225 in that given tuple. The seed data 210 maybe appropriate for training the semantic parser subsystem 114; duringoperation of the dialog system 100, the semantic parser subsystem 114,also referred to as the semantic parser 114, takes an utterance 225 asinput and determines a logical form 235. However, in some embodiments,the seed data 210 need not, but can, be used to directly train thesemantic parser 114.

The conversion subsystem 240 may include one or more of the ML models ofthe dialog system 100, and the training system 150 may be enabled totrain each of such ML models to enable those ML models to operate moreeffectively in the dialog system 100. For instance, as described belowin detail, the conversion subsystem 240 may include the ASR 108, thesemantic parser 114, and the TTS subsystem 120 of a dialog system 100,and in that case, the training system 150 may train each of the ASR 108,the semantic parser 114, and the TTS subsystem 120 as described herein.

In the conversion subsystem 240, one or more ML models of a dialogsystem 100 translate the seed data 210 into a converted version of theseed data 210, and the training system 150 utilizes an objectivefunction 250 to compare the converted version to the original seed data210. More specifically, in some embodiments, the conversion subsystem240 takes as input an original logical form 235 a from the seed data 210and generates a converted logical form 235 b by translating the originallogical form 235 a to one or more different formats (e.g., an utterance225 or speech) and then back to the a logical form 235. The translationsmay be performed by one or more ML models of the dialog system 100. Assuch, the converted logical form 235 b represents the original logicalform 235 a potentially with errors introduced through processing by theML models in the dialog system 100. The conversion subsystem 240 mayoperate on each original logical form 235 a in the seed data 210, thusenabling the training system 150 to train the ML models in theconversion subsystem 240 based on the resulting converted logical forms235 b.

Ideally, because the converted logical form 235 b is a translation, eachconverted logical form 235 b should be the same as the correspondingoriginal logical form 235 a from the seed data 210. However, this maynot be the case due to the introduction of errors by the ML models inthe conversion subsystem 240. In some embodiments, the training system150 utilizes a difference between the original logical form 235 a andthe converted logical form 235 b to train the ML models to behave betterin the context of the pipeline of the dialog system 100. Specifically,the training system 150 may apply an objective function 250 (i.e., aloss function) to each converted logical form 235 b and itscorresponding original logical form 235 a to determine a training value.Together, a stream or set of training values, determined based onperforming the above operations for the various original logical forms235 a from the seed data 210, form a training signal. In someembodiments, the training system 150 utilizes the training signal totrain the ML models participating in the conversion subsystem 240.

FIG. 3 is a flow diagram of a method 300 of using backpropagation totrain one or more ML models of a dialog system 100, according to certainembodiments. In some embodiments, prior to execution of this method 300,each ML model to be used by the conversion subsystem 240 of the trainingsystem 150 may have been, but need not have been, trained individuallyfor use in the dialog system 100. For example, if the conversionsubsystem 240 includes a TTS subsystem 120, then the TTS subsystem 120has been trained to map utterances 225 to audio data 455, such asthrough the use of training data that includes utterances 225 and theircorresponding audio data 455; if the conversion subsystem 240 includesan ASR 108, then the ASR 108 has been trained to map speech input 104 toutterances 225, such as through the use of training data that includesaudio data 455 (e.g., speech input 104) and corresponding utterances225; and if the conversion subsystem 240 includes a semantic parser 114,then the semantic parser 114 has been trained to map utterances 225 tological forms, such as through the use of training data that includesutterances 225 and corresponding logical forms 235. Alternatively,training via backpropagation as performed by the training system 150described herein may be used in lieu of training each modelindividually.

The method 300 depicted in FIG. 3, as well as other methods describedherein, may be implemented in software (e.g., as code, instructions, orprograms) executed by one or more processing units (e.g., processors orprocessor cores), in hardware, or in combinations thereof. The softwaremay be stored on a non-transitory storage medium, such as on a memorydevice. This method 300 is intended to be illustrative and non-limiting.Although FIG. 3 depicts various activities occurring in a particularsequence or order, this is not intended to be limiting. In certainembodiments, for instance, the activities may be performed in adifferent order, or one or more activities of the method 300 may beperformed in parallel. In certain embodiments, the method 300 may beperformed by the training system 150.

As shown in FIG. 3, at block 305, the training system 150 accesses seeddata 210 for use in the training system 150. The seed data 210 mayinclude a set of tuples, each tuple including an original utterance 225as well as an original logical form 235 a corresponding to the originalutterance 225. It will be understood that various techniques may be usedto collect the seed data 210, and such techniques may be manual,automatic, or a combination of both.

In some embodiments, the collection of the seed data 210 may be, atleast in part, a manual process. For instance, the seed data 210 may becrowdsourced. In some embodiments, a team of one or more individualsmanually writes a grammar to describe a structure of the logical form235. The team may generate a set of original logical forms 235 a to beincluded in the seed data 210 and may provide such original logicalforms 235 a to one or more individuals in a crowd, asking the crowd toconvert each original logical form 235 a into a corresponding originalutterance 225. Further, the team may provide an intermediate form torepresent each such original logical form 235 a, where the intermediateform is an abrupt or choppy variation of natural language that isrelatively simply to produce by the team and relatively simple tounderstand by the crowd. The intermediate form may assist the crowd inconversion because use of the intermediate form means the crowd need notlearn the language of logical forms 235. Thus, to convert an originallogical form into natural language (i.e., a corresponding originalutterance 225), the crowd may convert the corresponding intermediateform into natural language. It will be understood that multipleutterances 225 can equate to a common logical form 235, and thus, thecrowd may generate one or more original utterances 225 based on anoriginal logical form 235 a, and each such original utterance 225 may becombined in a respective tuple with the original logical form 235 a.

In another embodiment, a set of original utterances 225 are provided,and one or more individuals determine a logical form 235 for each suchoriginal utterance 225. However, it may provide more efficient to startwith a logical form 235 because individuals are likely more familiarwith natural language and may thus make faster work of generatingutterances 225 in natural language as compared to generating logicalforms 235 in a less familiar language.

Block 310 beings an iterative loop in which each tuple of the seed data210 is considered in turn. The training system 150 may iterate over thetuples in the seed data 210. With each iteration, one or more ML modelsof the dialog system 100 that are included in the conversion subsystem240 may be further tuned to provide accurate output. As described aboveand described further below, for each tuple of the seed data 210, anembodiment of the training system 150 utilizes the conversion subsystem240 to translate the tuple to a converted tuple. For instance, this mayinclude converting the utterance 225 of the tuple to a convertedutterance 225 or converting the original logical form 235 a of the tupleto a converted logical form 235 b, as in the example of FIG. 2. Thetraining system 150 then compares the converted tuple to the originaltuple to train the ML models in the conversion subsystem 240.Specifically, at block 310, the training system 150 selects from theseed data 210 a tuple that has not yet been considered so as to performthese activities.

At block 315, the conversion subsystem 240 of the training system 150applies the one or more ML models of the conversion subsystem 240 to theselected tuple, which was selected to block 310, to translate theselected tuple to a converted tuple. As described above, each ML modelused by the conversion subsystem 240 may perform a type of translation.Thus, ideally, the converted tuple is equal to the selected tuple.

At block 320, the training system 150 compares the converted tuple,determined at block 315, to the selected tuple, determined at block 310.For instance, the training system 150 may apply an objective function250 to perform this comparison. In some embodiments, utilizing theobjective function 250 or another technique, the training system 150determines a degree of error between the converted tuple and theselected tuple, where the selected tuple is the desired result of theconversions performed by the conversion subsystem 240.

At block 325, the training system 150 trains the ML models used by theconversion subsystem 240, or a subset of these ML models, usingbackpropagation based on the result of the comparison performed at block320. For instance, for an ML model implemented as a neural network, thetraining system 150 updates the weights of the nodes of such ML modelbased on the degree of error determined above.

At decision block 330, the training system 150 determines whether anytuples remain for consideration in the seed data 210. If one or moretuples have not yet been considered, then the method 300 returns toblock 310 to select another tuple. However, if all such tuples have beenconsidered, then the method 300 ends at block 335, with the ML models ofthe dialog system 100 having been trained and being useable in thedialog system 100.

FIG. 4 is a diagram of another example of a training system 150configured to train one or more ML models of a dialog system 100,according to certain embodiments. In some embodiments, the trainingsystem 150 is implemented as a computing device or portion thereof, suchas a server. The training system 150 may be implemented as a specializedhardware device or as program code, or a combination of both. Forinstance, the operations described herein as being performed by thetraining system 150 may be embodied in program code implementing thetraining system 150, where such program code is executable by one ormore processing units.

As described above, the conversion subsystem 240 of the training system150 includes one or more ML models selected from the dialog system 100,thus enabling the training system 150 to train those ML models using atraining signal based on comparing their output (i.e., a convertedlogical form 235 b) to expected output (i.e., an original logical form235 a from the seed data 210). The conversion subsystem 240 applies eachsuch ML model to tuples of seed data 210, as described further below, todetermine the training signal. Specifically, in the example of FIG. 4,the conversion subsystem 240 includes the ASR 108, the semantic parser114, and the TTS subsystem 120 of a dialog system 100. In someembodiments, the ASR 108, the semantic parser 114, and the TTS subsystem120 may be selected from a single dialog system 100 that utilizes thisspecific ASR 108, semantic parser 114, and TTS subsystem 120. Thetraining described herein performed by the training system 150 may be inaddition to conventional training, in which each such ML modelparticipating in the conversion subsystem 240 may be trained on anindividual basis, or in lieu of conventional training.

As shown in FIG. 4, the training system 150 may have access to a set ofseed data 210, which may include a set of tuples, each tuple includingan utterance 225 and a corresponding original logical form 235 a.Logical forms 235, such as the original logical form 235 a, may besyntactical expressions complying with a predefined grammar that isparseable by a dialog manager subsystem 116 of the dialog system 100.Thus, the original logical form 235 a in a tuple may be a structuredtranslation of the corresponding utterance 225 in the tuple. The seeddata 210 may be appropriate for training the semantic parser 114, which,during operation of the dialog system 100, takes an utterance 225 asinput and determines a logical form 235. However, in some embodiments,the seed data 210 need not, but can, be used to directly train thesemantic parser 114. As described below, the conversion subsystem 240may apply the one or more ML models participating in the conversionsubsystem 240 to tuples of the seed data 210 to convert those tuples inorder to determine a training signal.

In some embodiments, as in this example, the conversion subsystem 240 ofthe training system 150 may apply an inverse seq2seq model 410 to theoriginal logical form 235 a of each tuple in the seed data 210 to causethe inverse seq2seq model 410 to determine a second utterance 225corresponding to the original logical form 235 a. The second utterance225 may thus be a translation of the logical form 235 selected from thetuple of the seed data 210. In some embodiments, the inverse seq2seqmodel 410 is trained in parallel with the semantic parser 114, asdescribed herein, to be the inverse of the semantic parser 114, whichmay be a seq2seq model. For instance, the semantic parser 114 inputsutterances 225 and outputs logical forms 235, whereas the inverseseq2seq model 410 inputs logical forms 235 and thus outputs utterances225. More specifically, as trained herein, when provided with a logicalform 235 output by the semantic parser 114 based on a specific utterance225, the inverse seq2seq model 410 would output that same specificutterance 225. The second utterance 225 may be a textual translation ofthe original logical form 235 a but may include errors based on thepotential inaccuracy in the inverse seq2seq model 410 and, as such,based on the potential inaccuracy in the semantic parser 114.

In some embodiments, the conversion subsystem 240 of the training system150 applies the TTS subsystem 120 to the second utterance 225, asdetermined by the inverse seq2seq model 410, to cause the TTS subsystem120 to determine audio data 455 corresponding to the second utterance225 and thus to the original logical form 235 a. The audio data 455 maythus be an audio translation of the original logical form 235 a;however, the audio data 455 may incorporate errors introduced by theinverse seq2seq model 410 or the TTS subsystem 120. These would be thesame types of errors as would be introduced during operation of thedialog system 100 because the TTS subsystem 120 is part of the dialogsystem 100 and because the inverse seq2seq model 410 is a representationof the semantic parser 114, which is also part of the dialog system 100.

In some embodiments, the conversion subsystem of the training system 150applies the ASR 108 to the audio data 455 to cause the ASR 108 determinea third utterance 225, which corresponds to the audio data 455, thesecond utterance 225, and the original logical form 235 a. The thirdutterance 225 may thus be a textual translation of the original logicalform 235 a; however, the third utterance 225 may incorporate errorsintroduced by the inverse seq2seq model 410, the TTS subsystem 120, orthe ASR 108. These would be the same types of errors as would beintroduced during operation of the dialog system 100 because the TTSsubsystem 120 and the ASR 108 are part of the dialog system 100 andbecause the inverse seq2seq model 410 is a representation of thesemantic parser 114, which is also part of the dialog system 100.

In some embodiments, the conversion subsystem 240 of the training system150 applies the semantic parser 114 to the third utterance 225 to causethe semantic parser 114 to determine the converted logical form 235 b.The converted logical form 235 b may thus be a translation of theoriginal logical form 235 a; however, the converted logical form 235 bmay incorporate errors introduced by the inverse seq2seq model 410, theTTS subsystem 120, the ASR 108, or the semantic parser 114. These wouldbe the same types of errors as would be introduced during operation ofthe dialog system 100 because the TTS subsystem 120, the ASR 108, andthe semantic parser 114 are part of the dialog system 100 and becausethe inverse seq2seq model 410 is a representation of the semantic parser114, which is part of the dialog system 100.

Ideally, because each of the inverse seq2seq model 410, the TTSsubsystem 120, the ASR 108, and the semantic parser 114 is an ML modellearning to translate data from one form into another form, a convertedlogical form 235 b should be the same as the corresponding originallogical form 235 a selected from the seed data 210. However, this maynot be the case due to the introduction of errors by the inverse seq2seqmodel 410, the TTS subsystem 120, the ASR 108, and the semantic parser114. In some embodiments, the training system 150 utilizes a differencebetween the converted logical form 235 b and the corresponding originallogical form 235 a to train the TTS subsystem 120, the ASR 108, and thesemantic parser 114 to behave better in the context of the pipeline ofthe dialog system 100. In other words, the training system 150 may teachthe TTS subsystem 120, the ASR 108, and the semantic parser 114 togenerate a more accurate converted logical form 235 b, thus tuning theTTS subsystem 120, the ASR 108, and the semantic parser 114 to operatetogether with a reduction in errors.

To train the TTS subsystem 120, the ASR 108, and the semantic parser,the training system 150 may apply an objective function 250 (i.e., aloss function) to the converted logical form 235 b and the correspondingoriginal logical form 235 a to determine a training value. Together, astream or set of training values determined based on the variousoriginal logical forms 235 a in the seed data 210 form a trainingsignal. In some embodiments, the training system 150 utilizes thetraining signal to train one or more of (e.g., each of) the inverseseq2seq model 410, the TTS subsystem 120, the ASR 108, and the semanticparser 114. As a result, the TTS subsystem 120, the ASR 108, and thesemantic parser 114 learn to operate within the pipeline of the dialogsystem 100 and may thus more effectively work together.

FIG. 5 is a diagram of another example of a method 500 of usingbackpropagation to train one or more ML models of a dialog system 100,according to certain embodiments. Specifically, in this example, theconversion subsystem 240 applies the ASR 108, the semantic parser 114,and the TTS subsystem 120, and the training system 150 trains the ASR108, the semantic parser 114, and the TTS subsystem 120 usingbackpropagation.

In some embodiments, prior to execution of this method 500, each ofthese ML models may have been, but need not have been, trainedindividually. For instance, given the conversion subsystem 240 in theexample of FIG. 4, the inverse seq2seq model 410 has been trainedindividually to map logical forms 235 to utterances 225, such as throughthe use of training data that includes logical forms 235 and theircorresponding utterances 225, where the logical forms 235 are used astraining input and the corresponding utterances are the expected outputfor training; the TTS subsystem 120 has been trained to map utterances225 to audio data 455, such as through the use of training data thatincludes utterances 225 and their corresponding audio data 455; the ASR108 has been trained to map speech input 104 to utterances 225, such asthrough the use of training data that includes audio data 455 (e.g.,speech input 104) and corresponding utterances 225; and the semanticparser 114 has been trained to map utterances 225 to logical forms, suchas through the use of the same training data used to train the inverseseq2seq model 410 but with the utterances 225 as training input and thelogical forms 235 as the expected output for training. Alternatively,training via backpropagation as performed by the training system 150described herein may be used in lieu of training each modelindividually.

The method 500 depicted in FIG. 5, as well as other methods describedherein, may be implemented in software (e.g., as code, instructions, orprograms) executed by one or more processing units (e.g., processors orprocessor cores), in hardware, or in combinations thereof. The softwaremay be stored on a non-transitory storage medium, such as on a memorydevice. This method 500 is intended to be illustrative and non-limiting.Although FIG. 5 depicts various activities occurring in a particularsequence or order, this is not intended to be limiting. In certainembodiments, for instance, the activities may be performed in adifferent order, or one or more activities of the method 500 may beperformed in parallel. In certain embodiments, the method 500 may beperformed by the training system 150.

As shown in FIG. 5, at block 505, the training system 150 accesses seeddata 210 for use in the training system 150. The seed data 210 mayinclude a set of tuples, each tuple including an original utterance 225as well as an original logical form 235 a corresponding to the originalutterance 225. It will be understood that various techniques may be usedto collect the seed data 210, and such techniques may be manual,automatic, or a combination of both.

In some embodiments, the collection of the seed data 210 may be, atleast in part, a manual process. For instance, the seed data 210 may becrowdsourced. In some embodiments, a team of one or more individualsmanually writes a grammar to describe a structure of the logical form235. The team may generate a set of original logical forms 235 a to beincluded in the seed data 210 and may provide such original logicalforms 235 a to one or more individuals in a crowd, asking the crowd toconvert each original logical form 235 a into a corresponding originalutterance 225. Further, the team may provide an intermediate form torepresent each such original logical form 235 a, where the intermediateform is an abrupt or choppy variation of natural language that isrelatively simply to produce by the team and relatively simple tounderstand by the crowd. The intermediate form may assist the crowd inconversion because use of the intermediate form means the crowd need notlearn the language of logical forms 235. Thus, to convert an originallogical form into natural language (i.e., a corresponding originalutterance 225), the crowd may convert the corresponding intermediateform into natural language. It will be understood that multipleutterances 225 can equate to a common logical form 235, and thus, thecrowd may generate one or more original utterances 225 based on anoriginal logical form 235 a, and each such original utterance 225 may becombined in a respective tuple with the original logical form 235 a.

In another embodiment, a set of original utterances 225 are provided,and one or more individuals determine a logical form 235 for each suchoriginal utterance 225. However, it may provide more efficient to startwith a logical form 235 because individuals are likely more familiarwith natural language and may thus make faster work of generatingutterances 225 in natural language as compared to generating logicalforms 235 in a less familiar language.

Block 510 beings an iterative loop in which each tuple of the seed data210 is considered in turn. The training system 150 may iterate over thetuples in the seed data 210. With each iteration, one or more ML modelsof the dialog system 100 may be further tuned to provide accurateoutput. As described above and described further below, for each tupleof the seed data 210, an embodiment of the training system 150 utilizesthe conversion subsystem 240 to translate the tuple, specifically theoriginal logical form 235 a of the tuple, to a converted tuple,specifically to a converted logical form 235 b. The training system 150then compares the converted tuple to the original tuple to train the MLmodels in the conversion subsystem 240. Specifically, at block 510, thetraining system 150 selects from the seed data 210 a tuple that has notyet been considered so as to perform these activities.

At block 515, from the selected tuple, the training system 150 selectsthe original logical form 235 a. As described above, each tuple mayinclude an original logical form 235 a and corresponding utterance 225,and the training system 150 may select the original logical form 235 afrom among those.

At block 520, the training system 150 applies the inverse seq2seq model410 to the original logical form 235 a to cause the inverse seq2seqmodel to determine a second utterance 225. Ideally, the second utterance225 is the same as the original utterance 225 corresponding to theoriginal logical form 235 a in the seed data 210. However, this may notbe the case due potentially to errors in predictions. In the earlyiterations of the loop, the inverse seq2seq model 410 may performpoorly, for instance, outputting a random utterance 225 (e.g., a randomarrangement of words or letters) in the first iteration. In someembodiments, the inverse seq2seq model 410 improves as training proceedsover numerous iterations.

At block 525, the training system 150 applies the TTS subsystem 120 tothe second utterance 225 to cause the TTS subsystem 120 determine audiodata 455. The audio data 455 may be embodied in a sound file, such as a.wav file, for instance. Ideally, the audio data 455 is a perfecttranslation of the second utterance 225 and thus of the original logicalform 235 a. However, this may not be the case due potentially to errorsin predictions. In some embodiments, the TTS subsystem 120 may performpoorly in early iterations, for instance, outputting random audio (e.g.,a random arrangement of sounds or words) in the first iteration.Further, the second utterance 225 received by the TTS subsystem 120 mayincorporate an error introduced by the inverse seq2seq model 410, and insome embodiments, the TTS subsystem 120 bases its output on the outputof the inverse seq2seq model 410. Thus, the output of the TTS subsystem120 is impacted not only by its own history of learning, but also by thehistory of the inverse seq2seq model 410. In some embodiments, theinverse seq2seq model 410 and the TTS subsystem 120 improve as trainingproceeds over numerous iterations.

At block 530, the training system 150 applies the ASR 108 to the audiodata 455 to cause the ASR 108 to determine a third utterance 225.Ideally, the third utterance 225 is the same as the original utterance225 corresponding to the original logical form 235 a in the seed data210. However, this may not be the case due potentially to errors inpredictions. In the early iterations of this method 500, the ASR 108 mayperform poorly, for instance, outputting a random utterance 225 (e.g., arandom arrangement of words or letters) in the first iteration. Further,the audio data 455 received by the ASR 108 may incorporate an errorintroduced by the inverse seq2seq model 410 or the TTS subsystem 120,and in some embodiments, the ASR 108 bases its output on the output ofthe TTS subsystem 120. Thus, the output of the ASR 108 is impacted notonly by its own history of learning, but also by the histories of theinverse seq2seq model 410 and the TTS subsystem 120. In someembodiments, the inverse seq2seq model 410, the TTS subsystem 120, andthe ASR 108 improve as training proceeds over numerous iterations.

At block 535, the training system 150 applies the semantic parser 114 tothe third utterance 225 to cause the semantic parser 114 determine aconverted logical form 235 b. Ideally, the converted logical form 235 bis the same as the original logical form 235 a selected from the seeddata 210. However, this may not be the case due potentially to errors inpredictions. In the early iterations of this method 500, the ASR 108 mayperform poorly, for instance, outputting a random logical form 235(e.g., a random arrangement of words and symbols) in the firstiteration. Further, the third utterance 225 received by the semanticparser 114 may incorporate an error introduced by the inverse seq2seqmodel 410, the TTS subsystem 120, or the ASR 108, and in someembodiments, the semantic parser 114 bases its output on the output ofthe ASR 108. Thus, the output of the semantic parser 114 is impacted notonly by its own history of learning, but also by the histories of theinverse seq2seq model 410, the TTS subsystem 120, and the ASR 108. Insome embodiments, the inverse seq2seq model 410, the TTS subsystem 120,the ASR 108, and the semantic parser 114 improve as training proceedsover numerous iterations.

At block 540, an objective function 250 is applied to the convertedlogical form 235 b and the original logical form 235 a that was selectedfrom the selected tuple in the seed data 210. The objective function 250may compare its inputs to calculate a degree of difference between suchinputs, which are, in this case, the converted logical form 235 b andthe original logical form 235 a. Various techniques exist forconstructing an appropriate objective function 250 for the comparison oflogical forms 235, and one or more of such techniques may be used todevelop the objective function 250 used by the training system 150.

At block 545, the training system 150 trains the inverse seq2seq model410, the TTS subsystem 120, the ASR 108, and the semantic parser 114, ora subset of these models, using backpropagation based on the result ofthe objective function 250. In some embodiments, for instance, the TTSsubsystem 120 is implemented as a neural network, and the trainingsystem 150 updates the weights of the nodes of the TTS subsystem 120based on the training signal. Additionally or alternatively, the ASR 108is implemented as a neural network, and the training system 150 updatesthe weights of the nodes of the ASR 108 based on the training signal.Additionally or alternatively, the semantic parser 114 is implemented asa neural network, and the training system 150 updates the weights of thenodes of the semantic parser 114 based on the training signal.

At decision block 550, the training system 150 determines whether anytuples remain for consideration in the seed data 210. If one or moretuples have not yet been considered, then the method 500 returns toblock 510 to select another tuple. However, if all such tuples have beenconsidered, then the method 500 ends at block 555, with the ML models ofthe dialog system 100 having been trained and being useable in thedialog system 100.

In some embodiments, some of the operations of the method 500 describedabove can be skipped to provide backpropagation to a proper subset ofthe ML models selected from the dialog system 100 for inclusion in theconversion subsystem 240. For instance, from among the inverse seq2seqmodel 410, the TTS subsystem 120, the ASR 108, and the semantic parser114 in the conversion subsystem 240, the training system 150 may trainonly the inverse seq2seq model 410 and the semantic parser 114 or onlythe TTS subsystem 120 and the ASR 108.

Further, the conversion subsystem 240 is not limited to the example ofincluding the inverse seq2seq model 410, the TTS subsystem 120, the ASR108, and the semantic parser 114. For another example, the conversionsubsystem 240 may include only the TTS subsystem 120 and the ASR 108, inwhich case the training system 150 may utilize the conversion subsystem240 to convert the original utterance 225 to audio data 455 and then toa converted utterance 225, such that a comparison between originalutterances 225 and corresponding converted utterances is used to trainone or both of the TTS subsystem 120 and the ASR. Variousimplementations are within the scope of this disclosure.

FIG. 6 is a diagram of a distributed system 600 for implementing certainembodiments. In the illustrated embodiment, distributed system 600includes one or more client computing devices 602, 604, 606, and 608,coupled to a server 612 via one or more communication networks 610.Clients computing devices 602, 604, 606, and 608 may be configured toexecute one or more applications.

In various embodiments, server 612 may be adapted to run one or moreservices or software applications that enable the use of backpropagationto train ML models of the dialog system 100 as described herein. Forinstance, server 612 may execute some or all aspects of the trainingsystem 150 or some or all aspects of the dialog system 100.

In certain embodiments, server 612 may also provide other services orsoftware applications that can include non-virtual and virtualenvironments. In some embodiments, these services may be offered asweb-based or cloud services, such as under a Software as a Service(SaaS) model to the users of client computing devices 602, 604, 606,and/or 608. Users operating client computing devices 602, 604, 606,and/or 608 may in turn utilize one or more client applications tointeract with server 612 to utilize the services provided by thesecomponents. More specifically, for instance, each of client computingdevices 602, 604, 606, and/or 608 may be an embedded device configuredto execute the dialog system 100 and, further, configured to communicatewith server 612 to enable server 612 to train ML models of the dialogsystem 100 through backpropagation as described herein.

In the configuration depicted in FIG. 6, server 612 may include one ormore components 618, 620 and 622 that implement the functions performedby server 612. These components may include software components that maybe executed by one or more processors, hardware components, orcombinations thereof. It should be appreciated that various differentsystem configurations are possible, which may be different fromdistributed system 600. The embodiment shown in FIG. 6 is thus oneexample of a distributed system for implementing an embodiment systemand is not intended to be limiting.

Users may use client computing devices 602, 604, 606, and/or 608 tointeract with aspects of the dialog system 100 provided by server 612 inaccordance with the teachings of this disclosure. A client device mayprovide an interface (e.g., a speech interface) that enables a user ofthe client device to interact with the client device. The client devicemay also output information to the user via this interface. AlthoughFIG. 6 depicts only four client computing devices, any number of clientcomputing devices may be supported.

The client devices may include various types of computing systems suchas PA devices, portable handheld devices, general purpose computers suchas personal computers and laptops, workstation computers, wearabledevices, gaming systems, thin clients, various messaging devices,sensors or other sensing devices, and the like. These computing devicesmay run various types and versions of software applications andoperating systems (e.g., Microsoft Windows®, Apple Macintosh®, UNIX® orUNIX-like operating systems, Linux or Linux-like operating systems suchas Google Chrome™ OS) including various mobile operating systems (e.g.,Microsoft Windows Mobile®, iOS®, Windows Phone®, Android™, BlackBerry®,Palm OS®). Portable handheld devices may include cellular phones,smartphones, (e.g., an iPhone®), tablets (e.g., iPad®), personal digitalassistants (PDAs), and the like. Wearable devices may include GoogleGlass® head mounted display, and other devices. Gaming systems mayinclude various handheld gaming devices, Internet-enabled gaming devices(e.g., a Microsoft Xbox® gaming console with or without a Kinect®gesture input device, Sony PlayStation® system, various gaming systemsprovided by Nintendo®, and others), and the like. The client devices maybe capable of executing various different applications such as variousInternet-related apps, communication applications (e.g., E-mailapplications, short message service (SMS) applications) and may usevarious communication protocols.

Network(s) 610 may be any type of network familiar to those skilled inthe art that can support data communications using any of a variety ofavailable protocols, including without limitation TCP/IP (transmissioncontrol protocol/Internet protocol), SNA (systems network architecture),IPX (Internet packet exchange), AppleTalk®, and the like. Merely by wayof example, network(s) 610 can be a local area network (LAN), networksbased on Ethernet, Token-Ring, a wide-area network (WAN), the Internet,a virtual network, a virtual private network (VPN), an intranet, anextranet, a public switched telephone network (PSTN), an infra-rednetwork, a wireless network (e.g., a network operating under any of theInstitute of Electrical and Electronics (IEEE) 802.11 suite ofprotocols, Bluetooth®, and/or any other wireless protocol), and/or anycombination of these and/or other networks.

Server 612 may be composed of one or more general purpose computers,specialized server computers (including, by way of example, PC (personalcomputer) servers, UNIX® servers, mid-range servers, mainframecomputers, rack-mounted servers, etc.), server farms, server clusters,or any other appropriate arrangement and/or combination. Server 612 caninclude one or more virtual machines running virtual operating systems,or other computing architectures involving virtualization such as one ormore flexible pools of logical storage devices that can be virtualizedto maintain virtual storage devices for the server. In variousembodiments, server 612 may be adapted to run one or more services orsoftware applications that provide the functionality described in theforegoing disclosure.

The computing systems in server 612 may run one or more operatingsystems including any of those discussed above, as well as anycommercially available server operating system. Server 612 may also runany of a variety of additional server applications and/or mid-tierapplications, including HTTP (hypertext transport protocol) servers, FTP(file transfer protocol) servers, CGI (common gateway interface)servers, JAVA® servers, database servers, and the like. Exemplarydatabase servers include without limitation those commercially availablefrom Oracle®, Microsoft®, Sybase®, IBM® (International BusinessMachines), and the like.

In some implementations, server 612 may include one or more applicationsto analyze and consolidate data feeds and/or event updates received fromusers of client computing devices 602, 604, 606, and 608. As an example,data feeds and/or event updates may include, but are not limited to,Twitter® feeds, Facebook® updates or real-time updates received from oneor more third party information sources and continuous data streams,which may include real-time events related to sensor data applications,financial tickers, network performance measuring tools (e.g., networkmonitoring and traffic management applications), clickstream analysistools, automobile traffic monitoring, and the like. Server 612 may alsoinclude one or more applications to display the data feeds and/orreal-time events via one or more display devices of client computingdevices 602, 604, 606, and 608.

Distributed system 600 may also include one or more data repositories614, 616. These data repositories may be used to store data and otherinformation in certain embodiments. For example, one or more of datarepositories 614, 616 may be used to store seed data 210 or other datarequired to train ML models of the dialog system 100 by backpropagationas described herein. Data repositories 614, 616 may reside in a varietyof locations. For example, a data repository used by server 612 may belocal to server 612 or may be remote from server 612 and incommunication with server 612 via a network-based or dedicatedconnection. Data repositories 614, 616 may be of different types. Incertain embodiments, a data repository used by server 612 may be adatabase, for example, a relational database, such as databases providedby Oracle Corporation® and other vendors. One or more of these databasesmay be adapted to enable storage, update, and retrieval of data to andfrom the database in response to SQL-formatted commands.

In certain embodiments, one or more of data repositories 614, 616 mayalso be used by applications to store application data. The datarepositories used by applications may be of different types such as, forexample, a key-value store repository, an object store repository, or ageneral storage repository supported by a file system.

In certain embodiments, all or a portion of training ML models of thedialog system 100 by backpropagation, as described herein, may beoffered as services via a cloud environment. FIG. 7 is a diagram of acloud-based system environment in which training ML models of the dialogsystem 100 by backpropagation, as described herein, may be offered atleast in part as a cloud service, in accordance with certainembodiments. In the embodiment depicted in FIG. 7, cloud infrastructuresystem 702 may provide one or more cloud services that may be requestedby users using one or more client computing devices 704, 706, and 708.Cloud infrastructure system 702 may comprise one or more computersand/or servers that may include those described above for server 612.The computers in cloud infrastructure system 702 may be organized asgeneral purpose computers, specialized server computers, server farms,server clusters, or any other appropriate arrangement and/orcombination.

Network(s) 710 may facilitate communication and exchange of data betweenclient computing devices 704, 706, and 708 and cloud infrastructuresystem 702. Network(s) 710 may include one or more networks. Thenetworks may be of the same or different types. Network(s) 710 maysupport one or more communication protocols, including wired and/orwireless protocols, for facilitating the communications.

The embodiment depicted in FIG. 7 is only one example of a cloudinfrastructure system and is not intended to be limiting. It should beappreciated that, in some other embodiments, cloud infrastructure system702 may have more or fewer components than those depicted in FIG. 7, maycombine two or more components, or may have a different configuration orarrangement of components. For example, although FIG. 7 depicts threeclient computing devices, any number of client computing devices may besupported in alternative embodiments.

The term cloud service is generally used to refer to a service that ismade available to users on demand and via a communication network suchas the Internet by systems (e.g., cloud infrastructure system 702) of aservice provider. Typically, in a public cloud environment, servers andsystems that make up the cloud service provider's system are differentfrom the customer's own on-premises servers and systems. The cloudservice provider's systems are managed by the cloud service provider.Customers can thus avail themselves of cloud services provided by acloud service provider without having to purchase separate licenses,support, or hardware and software resources for the services. Forexample, a cloud service provider's system may host an application, anda user may, via the Internet, on demand, order and use the applicationwithout the user having to buy infrastructure resources for executingthe application. Cloud services are designed to provide easy, scalableaccess to applications, resources and services. Several providers offercloud services. For example, several cloud services are offered byOracle Corporation® of Redwood Shores, Calif., such as middlewareservices, database services, Java cloud services, and others.

In certain embodiments, cloud infrastructure system 702 may provide oneor more cloud services using different models such as under a Softwareas a Service (SaaS) model, a Platform as a Service (PaaS) model, anInfrastructure as a Service (IaaS) model, and others, including hybridservice models. Cloud infrastructure system 702 may include a suite ofapplications, middleware, databases, and other resources that enableprovision of the various cloud services.

A SaaS model enables an application or software to be delivered to acustomer over a communication network like the Internet, as a service,without the customer having to buy the hardware or software for theunderlying application. For example, a SaaS model may be used to providecustomers access to on-demand applications that are hosted by cloudinfrastructure system 702. Examples of SaaS services provided by OracleCorporation® include, without limitation, various services for humanresources/capital management, customer relationship management (CRM),enterprise resource planning (ERP), supply chain management (SCM),enterprise performance management (EPM), analytics services, socialapplications, and others.

An IaaS model is generally used to provide infrastructure resources(e.g., servers, storage, hardware and networking resources) to acustomer as a cloud service to provide elastic compute and storagecapabilities. Various IaaS services are provided by Oracle Corporation®.

A PaaS model is generally used to provide, as a service, platform andenvironment resources that enable customers to develop, run, and manageapplications and services without the customer having to procure, build,or maintain such resources. Examples of PaaS services provided by OracleCorporation® include, without limitation, Oracle Java Cloud Service(JCS), Oracle Database Cloud Service (DBCS), data management cloudservice, various application development solutions services, and others.

Cloud services are generally provided on an on-demand self-servicebasis, subscription-based, elastically scalable, reliable, highlyavailable, and secure manner. For example, a customer, via asubscription order, may order one or more services provided by cloudinfrastructure system 702. Cloud infrastructure system 702 then performsprocessing to provide the services requested in the customer'ssubscription order. For example, a customer may subscribe to informationservices or other services provided by the dialog system 100 inconversational form. Cloud infrastructure system 702 may be configuredto provide one or even multiple cloud services.

Cloud infrastructure system 702 may provide the cloud services viadifferent deployment models. In a public cloud model, cloudinfrastructure system 702 may be owned by a third party cloud servicesprovider and the cloud services are offered to any general publiccustomer, where the customer can be an individual or an enterprise. Incertain other embodiments, under a private cloud model, cloudinfrastructure system 702 may be operated within an organization (e.g.,within an enterprise organization) and services provided to customersthat are within the organization. For example, the customers may bevarious departments of an enterprise such as the Human Resourcesdepartment, the Payroll department, etc. or even individuals within theenterprise. In certain other embodiments, under a community cloud model,the cloud infrastructure system 702 and the services provided may beshared by several organizations in a related community. Various othermodels such as hybrids of the above mentioned models may also be used.

Client computing devices 704, 706, and 708 may be of different types(such as client computing devices 602, 604, 606, and 608 depicted inFIG. 6) and may be capable of operating one or more client applications.A user may use a client computing device to interact with cloudinfrastructure system 702, such as to request a service provided bycloud infrastructure system 702. An attacker may use a client device tosend malicious requests.

In some embodiments, the processing performed by cloud infrastructuresystem 702 may involve big data analysis. This analysis may involveusing, analyzing, and manipulating large data sets to detect andvisualize various trends, behaviors, relationships, etc. within thedata. This analysis may be performed by one or more processors, possiblyprocessing the data in parallel, performing simulations using the data,and the like. For example, big data analysis may be performed by cloudinfrastructure system 702 for providing training of ML models bybackpropagation as described herein. The data used for this analysis mayinclude structured data (e.g., data stored in a database or structuredaccording to a structured model) and/or unstructured data (e.g., datablobs (binary large objects)).

As depicted in the embodiment in FIG. 7, cloud infrastructure system 702may include infrastructure resources 730 that are utilized forfacilitating the provision of various cloud services offered by cloudinfrastructure system 702. Infrastructure resources 730 may include, forexample, processing resources, storage or memory resources, networkingresources, and the like.

In certain embodiments, to facilitate efficient provisioning of theseresources for supporting the various cloud services provided by cloudinfrastructure system 702 for different customers, the infrastructureresources 730 may be bundled into sets of resources or resource modules(also referred to as “pods”). Each resource module or pod may comprise apre-integrated and optimized combination of resources of one or moretypes. In certain embodiments, different pods may be pre-provisioned fordifferent types of cloud services. For example, a first set of pods maybe provisioned for a database service, a second set of pods, which mayinclude a different combination of resources than a pod in the first setof pods, may be provisioned for Java service, and the like. For someservices, the resources allocated for provisioning the services may beshared between the services.

Cloud infrastructure system 702 may itself internally use services 732that are shared by different components of cloud infrastructure system702 and that facilitate the provisioning of services by cloudinfrastructure system 702. These internal shared services may include,without limitation, a security and identity service, an integrationservice, an enterprise repository service, an enterprise managerservice, a virus scanning and white list service, a high availability,backup and recovery service, service for enabling cloud support, anemail service, a notification service, a file transfer service, and thelike.

Cloud infrastructure system 702 may comprise multiple subsystems. Thesesubsystems may be implemented in software, or hardware, or combinationsthereof. As depicted in FIG. 7, the subsystems may include a userinterface subsystem 712 that enables users or customers of cloudinfrastructure system 702 to interact with cloud infrastructure system702. User interface subsystem 712 may include various differentinterfaces such as a web interface 714, an online store interface 716where cloud services provided by cloud infrastructure system 702 areadvertised and are purchasable by a consumer, and other interfaces 718.For example, a customer may, using a client device, request (servicerequest 734) one or more services provided by cloud infrastructuresystem 702 using one or more of interfaces 714, 716, and 718. Forexample, a customer may access the online store, browse cloud servicesoffered by cloud infrastructure system 702, and place a subscriptionorder for one or more services offered by cloud infrastructure system702 that the customer wishes to subscribe to. The service request mayinclude information identifying the customer and one or more servicesthat the customer desires to subscribe to.

In certain embodiments, such as the embodiment depicted in FIG. 7, cloudinfrastructure system 702 may comprise an order management subsystem(OMS) 720 that is configured to process the new order. As part of thisprocessing, OMS 720 may be configured to: create an account for thecustomer, if not done already; receive billing and/or accountinginformation from the customer that is to be used for billing thecustomer for providing the requested service to the customer; verify thecustomer information; upon verification, book the order for thecustomer; and orchestrate various workflows to prepare the order forprovisioning.

Once properly validated, OMS 720 may then invoke an order provisioningsubsystem (OPS) 724 that is configured to provision resources for theorder including processing, memory, and networking resources. Theprovisioning may include allocating resources for the order andconfiguring the resources to facilitate the service requested by thecustomer order. The manner in which resources are provisioned for anorder and the type of the provisioned resources may depend upon the typeof cloud service that has been ordered by the customer. For example,according to one workflow, OPS 724 may be configured to determine theparticular cloud service being requested and identify a number of podsthat may have been pre-configured for that particular cloud service. Thenumber of pods that are allocated for an order may depend upon thesize/amount/level/scope of the requested service. For example, thenumber of pods to be allocated may be determined based upon the numberof users to be supported by the service, the duration of time for whichthe service is being requested, and the like. The allocated pods maythen be customized for the particular requesting customer for providingthe requested service.

Cloud infrastructure system 702 may send a response or notification 744to the requesting customer to indicate when the requested service is nowready for use. In some instances, information (e.g., a link) may be sentto the customer that enables the customer to start using and availingthe benefits of the requested services.

Cloud infrastructure system 702 may provide services to multiplecustomers. For each customer, cloud infrastructure system 702 isresponsible for managing information related to one or more subscriptionorders received from the customer, maintaining customer data related tothe orders, and providing the requested services to the customer. Cloudinfrastructure system 702 may also collect usage statistics regarding acustomer's use of subscribed services. For example, statistics may becollected for the amount of storage used, the amount of datatransferred, the number of users, and the amount of system up time andsystem down time, and the like. This usage information may be used tobill the customer. Billing may be done, for example, on a monthly cycle.

Cloud infrastructure system 702 may provide services to multiplecustomers in parallel. Cloud infrastructure system 702 may storeinformation for these customers, including possibly proprietaryinformation. In certain embodiments, cloud infrastructure system 702comprises an identity management subsystem (IMS) 728 that is configuredto manage customers information and provide the separation of themanaged information such that information related to one customer is notaccessible by another customer. IMS 728 may be configured to providevarious security-related services such as identity services, such asinformation access management, authentication and authorizationservices, services for managing customer identities and roles andrelated capabilities, and the like.

FIG. 8 is a diagram of an example computer system 800 that may be usedto implement certain embodiments. For example, in some embodiments,computer system 800 may be used to implement any of systems, subsystems,and components described herein. For example, multiple host machines mayprovide and implement training of ML models of the dialog system 100 bybackpropagation as described herein. Computer systems such as computersystem 800 may be used as host machines. As shown in FIG. 8, computersystem 800 includes various subsystems including a processing subsystem804 that communicates with a number of other subsystems via a bussubsystem 802. These other subsystems may include a processingacceleration unit 806, an I/O subsystem 808, a storage subsystem 818,and a communications subsystem 824. Storage subsystem 818 may includenon-transitory computer-readable storage media including storage media822 and a system memory 810.

Bus subsystem 802 provides a mechanism for letting the variouscomponents and subsystems of computer system 800 communicate with eachother as intended. Although bus subsystem 802 is shown schematically asa single bus, alternative embodiments of the bus subsystem may utilizemultiple buses. Bus subsystem 802 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, a local bus using any of a variety of bus architectures, and thelike. For example, such architectures may include an Industry StandardArchitecture (ISA) bus, Micro Channel Architecture (MCA) bus, EnhancedISA (EISA) bus, Video Electronics Standards Association (VESA) localbus, and Peripheral Component Interconnect (PCI) bus, which can beimplemented as a Mezzanine bus manufactured to the IEEE P1386.1standard, and the like.

Processing subsystem 804 controls the operation of computer system 800and may comprise one or more processors, application specific integratedcircuits (ASICs), or field programmable gate arrays (FPGAs). Theprocessors may include be single core or multicore processors. Theprocessing resources of computer system 800 can be organized into one ormore processing units 832, 834, etc. A processing unit may include oneor more processors, one or more cores from the same or differentprocessors, a combination of cores and processors, or other combinationsof cores and processors. In some embodiments, processing subsystem 804can include one or more special purpose co-processors such as graphicsprocessors, digital signal processors (DSPs), or the like. In someembodiments, some or all of the processing units of processing subsystem804 can be implemented using customized circuits, such as applicationspecific integrated circuits (ASICs), or field programmable gate arrays(FPGAs).

In some embodiments, the processing units in processing subsystem 804can execute instructions stored in system memory 810 or oncomputer-readable storage media 822. In various embodiments, theprocessing units can execute a variety of programs or code instructionsand can maintain multiple concurrently executing programs or processes.At any given time, some or all of the program code to be executed can beresident in system memory 810 and/or on computer-readable storage media822 including potentially on one or more storage devices. Throughsuitable programming, processing subsystem 804 can provide variousfunctionalities described above. In instances where computer system 800is executing one or more virtual machines, one or more processing unitsmay be allocated to each virtual machine.

In certain embodiments, a processing acceleration unit 806 mayoptionally be provided for performing customized processing or foroff-loading some of the processing performed by processing subsystem 804so as to accelerate the overall processing performed by computer system800.

I/O subsystem 808 may include devices and mechanisms for inputtinginformation to computer system 800 and/or for outputting informationfrom or via computer system 800. In general, use of the term inputdevice is intended to include all possible types of devices andmechanisms for inputting information to computer system 800. Userinterface input devices may include, for example, a keyboard, pointingdevices such as a mouse or trackball, a touchpad or touch screenincorporated into a display, a scroll wheel, a click wheel, a dial, abutton, a switch, a keypad, audio input devices with voice commandrecognition systems, microphones, and other types of input devices. Userinterface input devices may also include motion sensing and/or gesturerecognition devices such as the Microsoft Kinect® motion sensor thatenables users to control and interact with an input device, theMicrosoft Xbox® 360 game controller, devices that provide an interfacefor receiving input using gestures and spoken commands. User interfaceinput devices may also include eye gesture recognition devices such asthe Google Glass® blink detector that detects eye activity (e.g.,“blinking” while taking pictures and/or making a menu selection) fromusers and transforms the eye gestures as inputs to an input device(e.g., Google Glass®). Additionally, user interface input devices mayinclude voice recognition sensing devices that enable users to interactwith voice recognition systems (e.g., Siri® navigator) through voicecommands.

Other examples of user interface input devices include, withoutlimitation, three dimensional (3D) mice, joysticks or pointing sticks,gamepads and graphic tablets, and audio/visual devices such as speakers,digital cameras, digital camcorders, portable media players, webcams,image scanners, fingerprint scanners, barcode reader 3D scanners, 3Dprinters, laser rangefinders, and eye gaze tracking devices.Additionally, user interface input devices may include, for example,medical imaging input devices such as computed tomography, magneticresonance imaging, position emission tomography, and medicalultrasonography devices. User interface input devices may also include,for example, audio input devices such as MIDI keyboards, digital musicalinstruments and the like.

In general, use of the term output device is intended to include allpossible types of devices and mechanisms for outputting information fromcomputer system 800 to a user or other computer. User interface outputdevices may include a display subsystem, indicator lights, or non-visualdisplays such as audio output devices, etc. The display subsystem may bea cathode ray tube (CRT), a flat-panel device, such as that using aliquid crystal display (LCD) or plasma display, a projection device, atouch screen, and the like. For example, user interface output devicesmay include, without limitation, a variety of display devices thatvisually convey text, graphics and audio/video information such asmonitors, printers, speakers, headphones, automotive navigation systems,plotters, voice output devices, and modems.

Storage subsystem 818 provides a repository or data store for storinginformation and data that is used by computer system 800. Storagesubsystem 818 provides a tangible non-transitory computer-readablestorage medium for storing the basic programming and data constructsthat provide the functionality of some embodiments. Storage subsystem818 may store software (e.g., programs, code modules, instructions) thatwhen executed by processing subsystem 804 provides the functionalitydescribed above. The software may be executed by one or more processingunits of processing subsystem 804. Storage subsystem 818 may alsoprovide a repository for storing data used in accordance with theteachings of this disclosure.

Storage subsystem 818 may include one or more non-transitory memorydevices, including volatile and non-volatile memory devices. As shown inFIG. 8, storage subsystem 818 includes a system memory 810 and acomputer-readable storage media 822. System memory 810 may include anumber of memories including a volatile main random access memory (RAM)for storage of instructions and data during program execution and anon-volatile read only memory (ROM) or flash memory in which fixedinstructions are stored. In some implementations, a basic input/outputsystem (BIOS), containing the basic routines that help to transferinformation between elements within computer system 800, such as duringstart-up, may typically be stored in the ROM. The RAM typically containsdata and/or program modules that are presently being operated andexecuted by processing subsystem 804. In some implementations, systemmemory 810 may include multiple different types of memory, such asstatic random access memory (SRAM), dynamic random access memory (DRAM),and the like.

By way of example, and not limitation, as depicted in FIG. 8, systemmemory 810 may load application programs 812 that are being executed,which may include various applications such as Web browsers, mid-tierapplications, relational database management systems (RDBMS), etc.,program data 814, and an operating system 816. By way of example,operating system 816 may include various versions of Microsoft Windows®,Apple Macintosh®, and/or Linux operating systems, a variety ofcommercially-available UNIX® or UNIX-like operating systems (includingwithout limitation the variety of GNU/Linux operating systems, theGoogle Chrome® OS, and the like) and/or mobile operating systems such asiOS, Windows® Phone, Android® OS, BlackBerry® OS, Palm® OS operatingsystems, and others.

In certain embodiments, software instructions or code implementingtraining of ML models of the dialog system 100 by backpropagation, asdescribed herein, may be executed in system memory 810.

Computer-readable storage media 822 may store programming and dataconstructs that provide the functionality of some embodiments.Computer-readable storage media 822 may provide storage ofcomputer-readable instructions, data structures, program modules, andother data for computer system 800. Software (programs, code modules,instructions) that, when executed by processing subsystem 804 providesthe functionality described above, may be stored in storage subsystem818. By way of example, computer-readable storage media 822 may includenon-volatile memory such as a hard disk drive, a magnetic disk drive, anoptical disk drive such as a CD ROM, DVD, a Blu-Ray® disk, or otheroptical media. Computer-readable storage media 822 may include, but isnot limited to, Zip® drives, flash memory cards, universal serial bus(USB) flash drives, secure digital (SD) cards, DVD disks, digital videotape, and the like. Computer-readable storage media 822 may alsoinclude, solid-state drives (SSD) based on non-volatile memory such asflash-memory based SSDs, enterprise flash drives, solid state ROM, andthe like, SSDs based on volatile memory such as solid state RAM, dynamicRAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, andhybrid SSDs that use a combination of DRAM and flash memory based SSDs.

In certain embodiments, storage subsystem 818 may also include acomputer-readable storage media reader 820 that can further be connectedto computer-readable storage media 822. Reader 820 may receive and beconfigured to read data from a memory device such as a disk, a flashdrive, etc.

In certain embodiments, computer system 800 may support virtualizationtechnologies, including but not limited to virtualization of processingand memory resources. For example, computer system 800 may providesupport for executing one or more virtual machines. In certainembodiments, computer system 800 may execute a program such as ahypervisor that facilitated the configuring and managing of the virtualmachines. Each virtual machine may be allocated memory, compute (e.g.,processors, cores), I/O, and networking resources. Each virtual machinegenerally runs independently of the other virtual machines. A virtualmachine typically runs its own operating system, which may be the sameas or different from the operating systems executed by other virtualmachines executed by computer system 800. Accordingly, multipleoperating systems may potentially be run concurrently by computer system800.

Communications subsystem 824 provides an interface to other computersystems and networks. Communications subsystem 824 serves as aninterface for receiving data from and transmitting data to other systemsfrom computer system 800. For example, communications subsystem 824 mayenable computer system 800 to establish a communication channel to oneor more client devices via the Internet for receiving and sendinginformation from and to the client devices.

Communication subsystem 824 may support both wired and/or wirelesscommunication protocols. For example, in certain embodiments,communications subsystem 824 may include radio frequency (RF)transceiver components for accessing wireless voice and/or data networks(e.g., using cellular telephone technology, advanced data networktechnology, such as 3G, 4G or EDGE (enhanced data rates for globalevolution), WiFi (IEEE 802.XX family standards, or other mobilecommunication technologies, or any combination thereof), globalpositioning system (GPS) receiver components, and/or other components.In some embodiments communications subsystem 824 can provide wirednetwork connectivity (e.g., Ethernet) in addition to or instead of awireless interface.

Communication subsystem 824 can receive and transmit data in variousforms. For example, in some embodiments, in addition to other forms,communications subsystem 824 may receive input communications in theform of structured and/or unstructured data feeds 826, event streams828, event updates 830, and the like. For example, communicationssubsystem 824 may be configured to receive (or send) data feeds 826 inreal-time from users of social media networks and/or other communicationservices such as Twitter® feeds, Facebook® updates, web feeds such asRich Site Summary (RSS) feeds, and/or real-time updates from one or morethird party information sources.

In certain embodiments, communications subsystem 824 may be configuredto receive data in the form of continuous data streams, which mayinclude event streams 828 of real-time events and/or event updates 830,that may be continuous or unbounded in nature with no explicit end.Examples of applications that generate continuous data may include, forexample, sensor data applications, financial tickers, networkperformance measuring tools (e.g. network monitoring and trafficmanagement applications), clickstream analysis tools, automobile trafficmonitoring, and the like.

Communications subsystem 824 may also be configured to communicate datafrom computer system 800 to other computer systems or networks. The datamay be communicated in various different forms such as structured and/orunstructured data feeds 826, event streams 828, event updates 830, andthe like to one or more databases that may be in communication with oneor more streaming data source computers coupled to computer system 800.

Computer system 800 can be one of various types, including a handheldportable device (e.g., an iPhone® cellular phone, an iPad® computingtablet, a PDA), a wearable device (e.g., a Google Glass® head mounteddisplay), a personal computer, a workstation, a mainframe, a kiosk, aserver rack, or any other data processing system. Due to theever-changing nature of computers and networks, the description ofcomputer system 800 depicted in FIG. 8 is intended only as a specificexample. Many other configurations having more or fewer components thanthe system depicted in FIG. 8 are possible. Based on the disclosure andteachings provided herein, a person of ordinary skill in the art willappreciate other ways and/or methods to implement the variousembodiments.

Although specific embodiments have been described, variousmodifications, alterations, alternative constructions, and equivalentsare possible. Embodiments are not restricted to operation within certainspecific data processing environments, but are free to operate within aplurality of data processing environments. Additionally, althoughcertain embodiments have been described using a particular series oftransactions and steps, it should be apparent to those skilled in theart that this is not intended to be limiting. Although some flowchartsdescribe operations as a sequential process, many of the operations canbe performed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process may have additional steps notincluded in the figure. Various features and aspects of theabove-described embodiments may be used individually or jointly.

Further, while certain embodiments have been described using aparticular combination of hardware and software, it should be recognizedthat other combinations of hardware and software are also possible.Certain embodiments may be implemented only in hardware, or only insoftware, or using combinations thereof. The various processes describedherein can be implemented on the same processor or different processorsin any combination.

Where devices, systems, components or modules are described as beingconfigured to perform certain operations or functions, suchconfiguration can be accomplished, for example, by designing electroniccircuits to perform the operation, by programming programmableelectronic circuits (such as microprocessors) to perform the operationsuch as by executing computer instructions or code, or processors orcores programmed to execute code or instructions stored on anon-transitory memory medium, or any combination thereof. Processes cancommunicate using a variety of techniques including but not limited toconventional techniques for inter-process communications, and differentpairs of processes may use different techniques, or the same pair ofprocesses may use different techniques at different times.

Specific details are given in this disclosure to provide a thoroughunderstanding of the embodiments. However, embodiments may be practicedwithout these specific details. For example, well-known circuits,processes, algorithms, structures, and techniques have been shownwithout unnecessary detail in order to avoid obscuring the embodiments.This description provides example embodiments only, and is not intendedto limit the scope, applicability, or configuration of otherembodiments. Rather, the preceding description of the embodiments willprovide those skilled in the art with an enabling description forimplementing various embodiments. Various changes may be made in thefunction and arrangement of elements.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that additions, subtractions, deletions, and other modificationsand changes may be made thereunto without departing from the broaderspirit and scope as set forth in the claims. Thus, although specificembodiments have been described, these are not intended to be limiting.Various modifications and equivalents are within the scope of thefollowing claims.

What is claimed is:
 1. A method comprising: accessing seed datacomprising training tuples, each training tuple comprising a respectivelogical form; selecting from the seed data a training tuple comprising alogical form; converting, using a sequence of two or more machinelearning (ML) models selected from a dialog system, the training tupleto a converted tuple; determining a training signal by applying anobjective function to compare the converted tuple and the trainingtuple; and training the two or more ML models of the dialog system viabackpropagation based on the training signal.
 2. The method of claim 1,wherein converting the training tuple to the converted tuple comprisesutilizing an automatic speech recognition (ASR) subsystem of the dialogsystem and a text-to-speech (TTS) subsystem of the dialog system.
 3. Themethod of claim 2, wherein each training tuple of the seed data furthercomprises a respective utterance corresponding to the respective logicalform of the training tuple, and wherein converting the training tuple tothe converted tuple further comprises: determining audio data byapplying the TTS subsystem to the utterance of the training tuple;determining a converted utterance by applying the ASR subsystem to theaudio data; and wherein the objective function is configured to comparethe converted utterance to the utterance from the training tuple.
 4. Themethod of claim 2, wherein converting the training tuple to theconverted training tuple further comprises utilizing a semantic parserof the dialog system and an inverse sequence-to-sequence model.
 5. Themethod of claim 4, wherein converting the training tuple to theconverted tuple further comprises: determining a first utterance byapplying the inverse seq2seq model to the logical form of the trainingtuple; determining audio data by applying the TTS subsystem to the firstutterance; determining a second utterance by applying the ASR subsystemto the audio data; and determining a converted logical form by applyingthe semantic parser to the second utterance, wherein the objectivefunction is configured to compare the converted logical form to thelogical form from the training tuple.
 6. The method of claim 5, furthercomprising incorporating the TTS subsystem, the ASR subsystem, and thesemantic parser into the dialog system.
 7. The method of claim 5,wherein the semantic parser is a sequence-to-sequence neural network,and wherein the inverse sequence-to-sequence model is an inverse of thesemantic parser.
 8. The method of claim 2, wherein the ASR subsystem isa sequence-to-sequence neural network.
 9. The method of claim 2, whereinthe TTS subsystem is a sequence-to-sequence neural network.
 10. Atraining system comprising a conversion subsystem configured to: inputtraining tuples of seed data, each training tuple of the seed datacomprising a respective logical form; and convert the training tuplesinto converted tuples by applying to the training tuples two or moremachine learning (ML) models selected from a dialog system; and anobjective function configured to compare the converted tuples to thetraining tuples to provide a training signal, wherein the trainingsystem trains the two or more ML models of the dialog system viabackpropagation using the training signal, wherein, after training, theone or more ML models of the conversion subsystem are incorporated intothe dialog system configured to provide a dialog with a user.
 11. Thetraining system of claim 10, wherein the conversion subsystem is furtherconfigured to: access a training tuple selected from the seed data, thetraining tuple comprising a logical form; determine audio data byapplying a text-to-speech (TTS) subsystem, selected from the dialogsystem, to an utterance corresponding to the logical form selected fromthe training tuple of the seed data; and determine a second utterance byapplying an automatic speech recognition (ASR) subsystem, selected fromthe dialog system, to the audio data.
 12. The training system of claim11, wherein: the training tuple further comprises an original utterance;and the objective function is further configured to compare the secondutterance to the original utterance from the training tuple.
 13. Thetraining system of claim 11, wherein the conversion system is furtherconfigured to: determine the utterance by applying an inversesequence-to-sequence model to the logical form of the training tuple;and determine a converted logical form by applying a semantic parser,selected from the dialog system, to the second utterance, wherein theobjective function is further configured to compare the convertedlogical form to the logical form from the training tuple.
 14. Thetraining system of claim 13, wherein the semantic parser is asequence-to-sequence neural network, and wherein the inversesequence-to-sequence model is an inverse of the semantic parser.
 15. Thetraining system of claim 11, wherein the ASR subsystem is asequence-to-sequence neural network.
 16. The training system of claim11, wherein the TTS subsystem is a sequence-to-sequence neural network.17. A computer-program product for training machine learning models of adialog system, the computer-program product comprising acomputer-readable storage medium having program instructions embodiedthereon, the program instructions executable by one or more processorsto cause the one or more processors to perform a method comprising:accessing seed data comprising training tuples, each training tuplecomprising a respective logical form; selecting from the seed data atraining tuple comprising a logical form; converting the logical form ofthe training tuple to a converted logical form by applying to thelogical form a text-to-speech (TTS) subsystem of a dialog system, anautomatic speech recognition (ASR) subsystem of the dialog system, and asemantic parser of the dialog system; determining a training signal byusing an objective function to compare the converted logical form to thelogical form; and training the TTS subsystem, the ASR subsystem, and thesemantic parser of the dialog system via backpropagation based on thetraining signal.
 18. The computer-program product of claim 17, whereinconverting the logical form to the converted logical form furthercomprises utilizing an inverse sequence-to-sequence model that is aninverse of the semantic parser.
 19. The computer-program product ofclaim 18, wherein converting the logical form to the converted logicalform further comprises: determining a first utterance by applying theinverse seq2seq model to the logical form of the training tuple;determining audio data by applying the TTS subsystem to the firstutterance; determining a second utterance by applying the ASR subsystemto the audio data; and determining the converted logical form byapplying the semantic parser to the second utterance.
 20. Thecomputer-program product of claim 18, wherein each of the semanticparser and the inverse sequence-to-sequence model is a respectivesequence-to-sequence neural network.